There are a number of prior techniques for deriving a high-resolution image from a plurality of low-resolution images.
In one technique, each low resolution image is aligned to a reference image using an alignment algorithm such as Lucas-Kanade [1-5]. The aligned images are then combined using stacking (robust sum), Bayesian inference, or learned statistics. There are two primary problems with this approach. (1) It attempts to achieve sub-pixel alignment accuracy in aligning the low-resolution images using only the low-resolution image. (2) This approach is not model-based, so it cannot accommodate barrel/pincushion distortion, diffraction or other effects.
In another technique, both the super-resolved image and the alignment parameters are constructed through optimization of the likelihood of the measured data (y) given the alignment parameters (A) and hypothesized super-resolved image (x) That is, the algorithm maximizes P(y|A, x). Some of these algorithms can optionally use a prior on the alignment parameters or the hypothesis (maximizing either P(y, x|A) or P(y, x, A). However is difficult (and frequently unstable) to simultaneously align and resolve the images.
An advantage of the model based approaches (optimization, Tipping-Bishop [6] and our own) is that the formulation is very general. For example, the set of “alignment parameters” (A) may capture any number of transformation parameters for example, degree of pin-cushion distortion, degree of barrel distortion, shift, rotation, degree of blurring kernels including Gaussian or other diffraction kernels,
US Patent Application Publication US2004/0170340 A1 of Tipping and Bishop refers to a Bayesian technique for computing a high resolution image from multiple low resolution images. The algorithm in the Tipping-Bishop application marginalizes the super-resolved image out of P(y, x|A) allowing one to directly optimize the likelihood for the alignment parameters followed by a super-resolution step. That is, the algorithm allows direct computation of P(y|A), allowing an optimization algorithm to directly optimize the alignment parameters. In the Tipping-Bishop application, these alignment parameters included shift, rotation and width of the point spread function (A=<s, θ, γ>) for the optical system (degree of blur). The problem with the approach of the Tipping-Bishop application is that it is mathematically incorrect. In the derivation of the approach they made a major algebra or formulation mistake with the result that the resulting alignment likelihood P(y|A) is incorrect. In practice, the algorithm frequently diverges when optimizing some imaging parameters, particularly the point spread function.